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abstract 

For instructional films to be e f fective teaching 
device, testina based on their subject material must be devised and 
used. Serial Multiple Discrimination Teachinq Examinations (Teachinq 
Tests or TT's) have been so used in an introductory psychology course 
attended by 5,000 students in the last five years. Since these tests 
have a low chance factor (the probability of aetting a single 
question correct by guessing can be as low as 1/6, 2 e 0), students do 
not guess. They study instead, and many of them have made perfect 
scores (40 consecutive errorless ouestions) . Studies have shovn that 
students taught by these examinations earn a higher proportion of 
correct answers and have a wider range of scores than those taught by 
other objective exams; they also scored above the mean on new exams 
over comparable material which were prepared by proiessors at other 
universities; and item analyses have consistently shown larger 
discrimination indices and higher internal consistency coefficients 
for TT's than for True False or Multiple Choice questions. (IF) 
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Somehow, the bright future that was envisioned for A-V materials some 25 
years ago has never materialized • One of the reasons for that failure may be 
laid direotly to outmoded researoh studies whioh state that the approved 
method for using a film is to "introduce it, soreen it, disouss it, and then 
show it again to answer all the questions," Instead of trying to find effeotlve 
ways to help aohleve the wider oourse objeotivws, these researoh studies 
attempted to justify the use of more film time, when, as evidenoed by the pro- 
portion of film questions included on the final examination, their contribution 
war. not important. 

College textbook publishers havo been foroed to provide objective exam- 
inations to buoy teachers as one of the ohief inducements for using their books. 
Repeat adoptions are frequently lost, not because the text has gone out of date, 
but beoause the store of questions has been exhausted, If p ovidlng reliablo 
and valid questions is essential to the achievement of speoifio educational 
objectives by college textbooks (to say nothing about increasing sales), why 
has no film ever been aooompanled by a validated set of objeotivs examination 
questions? It looks as if film producsrs have little faith in their product's 
ability to e cm tribute to an approved educational objective* 



But why has no further research been published? Without documenting it 
now, might 1 suggest that it is actually hazardous to a young teacher's career 
to do research on olasaroea teaching, and doubly dangerous when films are in- 
volved, because the threats come from both students and other faculty meabers, 
In addition, if his research findings are critical of the traditional views of 
the professional journals, few editors will publish them. 

Ho Mtter how relevant, interesting, pedagogically sound, or technically 
perfect a film production may be, it is of no value unless it is used. Its use 
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must also be followed by significant changes in tho desired direction in 
the measurable behavior of the learner* These ohanges cannot be determined 
without an additional expenditure of time and effort on the part of the 
teaoher. The most important aspeot of nearly any oollege course has always 
been the examination. Pressey ( ) demonstrated that regular examinations 

have positive teaching values, but Skinner ( ) has maintained that they are 

usually infrequent and aversive. Although I have successfully used different 
adaptations of the Pressey self-scoring teaohlng test techniques, the 
objeotlonable hardware was not eliminated until 40 years later when the 
principle of the latin square method was adapted for a paper and penoil test. 
Let me desoribe how we have been able to inorease the value of films for 
more than 5*000 students during the last five years with the help of a new 
instructional device, the Serial Multiple Discrimination Teaohlng Examinations, 
or simply, Teaching Tests (TT). 

Depending upon the seating capaolty of the room, I now teach one intro- 
ductory psychology olass of 350 - 500 students each quarter i I cover the 
oontent of a larger, more ooraplete text in 50 olass sessions of one quarter 
than when the course extended over a full aoademio yeari the level of achieve- 
ment is now 50 to 100 per cent higher than it has ever been previously i and 
in addition, I new screen and examine ovar more than 50 reels of films, store 
than 100 slides of a 300 slide series, and more than 100 overhead transparen - 
cies . One half of all ay exam questions are taken directly from A-V materials, 
with no loss of mastery of textbook Content, and the modal score on most ex ams 
is a per foot score. Because of severe budgetary restriotions, I an assigned 
only one graduate Teaching Assistant, must use biweekly instead of weekly 
exams, and 7 spend 15 hours in student conferences each week. 

Description of TT. A sample block of five TT questions and three special 



answer sheet fornats are presented on page 1 of the handout, Each such block 
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of five TT questions has five numbered true-false stems with four lists of 
five true matohing associations printed above them. On the speoial TT answei- 
sheetf you are to slash the + or - symbol underneath the first question number 
to indicate whether you think the atom is true or false i then slash the one 
letter in eaoh of the four columns of letters which corresponds to the olosest 
true matohing association in eaoh of the four lists of five lettered matching 
alternatives. (The oorreot answers for questions 1-5 ara slashed to lllus- 
trato the Verified Latin Square format, + E H L S +.) If you ohoose the 
oorreot matohing alternatives, the symbol in oolumn five opposite the row with 
no slashes will always verify your first tentative T-F answer to that question. 
Sinoe eaoh letter is only used onoe in a blook of five questions, if the 
answers to any four questions are all oorreot, the answers to the fifth ques- 
tion must be the one letter and symbol in eaoh oolumn that has not been used 
previously. 

Procedure. Students study TT questions of the VLS variety in their manual as 
a homework assignment before they see the film, and look for answers to the 
questions they had missed at home when it is soreened in olass. In order to 
eliminate rote memorising, and emphasise the association of related ideas, 
olass examinations are given over the sane or new material with the US format 
(questions 6-10 show new orders for both questions and matohing associations 
with the verification oolumn eliminated) or the LtB format (in questions 11-15 
the order of the matohii^ associations has been randomised). 

Teaohing Tests over films have been administered under various combinations 
of before and after screening, and before and after having studied the questions 
in a verified latln square format. The amount and immediaoy of feedbaok on the 
regular course exams was controlled and suooessively reduced by changing the 
order of the question parts, by using different VLS, US, and IRB blocks. In 
eplte of the low chance euccose factor (the probability of getting a air^le 
question correct solely by guesting can be as lew as 1 / 6 , 250 ) all five parts 
O 
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mu8t be correct to score a single point. Since the Teaching Test 
questions are prepared in blocks of five, if the student is not familiar 
with the TT technique and has no knowledge of the subject matter, the 
probability of answering any block of five questions correctly may be 
as low as 1 out of 9,536,813,164,062,500,000. If he is familiar with 
the technique but has no knowledge of the subject matter, the odds can 
still be as low as 1 out of 26,542,000 against him. With recognition 
that guessing simply cannot work in my classes, students do not try it. 
They study Instead, and many of them have made perfect scores (40 
consecutive errorless questions). 
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Results. Data from rows 1, 2, and 3 of Table 1 on page 2 reveal that new TT 
questions over leotures are harder than questions over the text or films i 
that new TT questions in row 4 are more diffioult than old familiar questions 
in row 5l that the same questions presented in randomised format are more 
diffioult than in either VLS or US formats (rows 6 - 10). When the identical 
film questions were presented agAln in a new and more diffioult format after 
aeeing the films (rows 11 and 12) , there was an absolute lnorease of 52 per- 
centage points in achievement, with only an 8 percentage point drop in the 
discrimination index, and none in the internal consistency coefficient. This 
result is contrary to what has been found with other types of objective exam 
questions. In order to inoreAse learning > all students should have ready 
pre-test aooees to all of the TT questions. The good students will master 
more questions at a faster rate than the slow students, resulting in high 
discrimination indices and high internal consistency coefficients, even though 
the average olass achievement hits unprecedented levols. 

Previous studies have shown that balanced experimental designs, where one 
student was given two types of questions or two formats on the same exam, 
ulways resulted in inflated t's because the students would spend most of their 
time on the question format that paid off with the highest scores. More con- 
servative data were obtained when different groups of individuals were assigned 
to a single, but different, examination format on successive weeks. When data 
from many different TT questions of the same kind were sums ted across several 
quarters, an interesting new finding appeared (see Table 2). Scores for the 
top criterion group on both new and familiar textbook and film questions de- 
crease with decreased feedback. Although *coros for the bottom criterion group 
also decreased on new questions, they aotually increased as less feedback was 
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provided on the old, familiar film questions. One possible explanation night 
be that the oollege students with poor verbal profioienoy were able to manipu- 
late the piotures more meaningfully than they oould the verbal symbols. This 
finding is especially significant for the teaohing of "disadvantaged" students. 
(The interoorrelations between the TT soores and the soores on the pre-oollege 
entranoe test battery suggest that high "reading comprehension" soores are more 
important than so-called "intelligence", and that reading speed is not an 
important factor because it oan be offset by high motivation and more study 
tine.) 

The data from Table 3 on the film Behavior Modification emphasises the 
importance c? feedback from the latin square format in answering complex 
questions. When the identical questions were presented again five weeks later 
(rows 2 ard 4), but without the oues from the latin square arrangement or the 
verifying synfcols, the top criterion group displayed the largest deoline we had 
witnessed (95 to 86 36 ), but the bottom group increased from 36 to 41 per cent 
oorreot. A subsequent analysis revealed that it was impossible to discriminate 
logioally between the matching associations for oertain questions once feedback 
was removed. The only way they could be answered oorreotly on the retest was 
to remember which associations had been reinforoed five weeks earlier with the 
latin squares. The discrimination index for these particularly difficult ques- 
tions was higher then for the others (54. vs. 29*). Questions whioh had been 
lnoluded on an US exam (row 4) were answered with 8 percentage points higher 
aoouraoy than those whioh had not been included (row 5)» 

The new and unambiguous questions covering much simpler concepts taken 
fro* reviews of the Tit l cut Follies and the film (Table 4) were answered with 
excellent accuracy on their very firat presentation (rows 1 and 4). With this 
relatively familiar and highly publicised subject matter, changing the examina- 
tion format for new questions to reduce feedback was more datrimental to the 
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bottom group of students than to the top oriterlon group. In the last two rows 
(9 and 10) the top oriterlon group deollned 1.8 percentage points while the 
bottom group lost 16.0 percentage points on the randomised format. 

Earlier studies have shown that students taught with the TT's earn a 
higher proportion of oorreot answers, and have a wider range of soores than on 
other objective exams i they also soored above the mean on new exams over com- 
parable material whioh were prepared by professors at other universities i and 
item analyses have consistently shown larger discrimination Indices and higher 
internal oonsistenoy ooeffioients for TT than for TF or MC questions. Technically 
the TT’s are not programed devices (although they oan be), but the results 
from one early study suggested that they were as effective as oertain programed 
texts The availability of a programed film enabled us to extend our jtudy to 
oompare the relative effectiveness of the Fields Teaohlng Tests and the Markle 
and Tiemann Programed Votes in learning the content of the programed film 
Programing is a Process. (See page 5). 

A conventional experimental design (Exhibit D) was selected, and tho Pro- 
gramed Notes and the training TT's distributed to a s ©looted half of the class, 

On the day after the film was soreened a 30 item TT and a 120 item TF exam were 
administered. Unknown to the students, the TF exam contained 30 questions over 
eaoh of four areas to help pinpoint experimental effects. The appropriate 
examinations were administered to four groups of 40 Psychology students and 
one group of 40 Engineering students. 

Fifty-six per cent of the group who had studied the training TT's made 
perfeot, scores on the TT exam aa opposed to seven per oent for the students who 
had studied the Program Notes. In spite of the very high average aohlevenent 
level (85 oorreot) for those who had atudied the training TT's, item analyses 
re *ealed significantly better average discrimination lndioes (29* to 11.) 
and internal oonsistenoy ooeffioients (.57 as opposed to ,22) for the TT 
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than for the TF items (Table 5, row? 3 and 5)» 

Table 6 shows that among the Psychology groups, the total test scores of 
those students who had offioially studied the TT'n were superior to those who 
used the Program Notes at the ,001 level of confidence. There was no signifi- 
cant difference betwoen the total soores of the Psychology students and the 
Engineers who had studied only the program notes .When the part soores on the 
TF test were analysed, the soores of the Pcyohology students who had studied 
the TT were significantly superior to those who had studied the Program Notes 
for all but the items which were taken direotly from th) film or notes and were 
not mentioned in the TT, Quite unexpectedly, the scores of the students who had 
studied the true matching associations in the training TT were signifioantly 
better for the 15 questions whloh had been ohanged from true to false. It was 
evident that these students had not rote nomorised the TT's, but were respond- 
ing to the larger ideas involved in the associations. 

Conclusion, As long as films are designed by produoers to appeal to as much 
of the kindergarden to adult epeotrum as possible, and used by teaohers to meet 
relatively low level entertainment objectives, they will not be utilised at 
anywhere near their true educational potential, But when films and Teaching 
Tests are used together, the quality of the learning olosely approximates that 
obtained with a good textbook, and far exceeds the expectations of many pro- 
duoers, We now know how to use films In order to teaoh more effeotively 
without the loss of precious olassroom tins, I hope that in the near future 
produoers will provide us with films timed U specific educstlonal age levels, 
tnd filled with sn abundance of up-to-the-minute datai that distributors 
will send complete scripts, and validated objective examination questions 
with each ssls and rental print | that rental libraries will kttp enough prints 
on hand to guarantse scheduling in the same week the textbook topio it assigned, 
if ordered six weeks in advance. The A-Y industry and professionals must also 
help busy teacher) measure the contribution of lie own produota to the learner. 
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Teaohing Teats Revolutionise Effeotlveneas of Classroom Films 

Paul B. Fields 
University of Washington 

Exhibit A. Illustrative Training Questions for Form E of Fields* Serial 
Multiple Discrimination Teaohing Examinations* 



A-Heijii Shrine, Tokyo 
B-Big Ben, London 
C»Elffel Tower, Paris 
D-Linooln Memorial, Washington D.C, 
E -Kremlin, Moscow 

F-Pennsylvania Avenue 
Q-Colshas 
H-Red Square 
X-Beatlea 

J-Left Bank of Seine 



K-Tea and orumpets 
L-Borvoh and eavlar 
M-Tempura and sake 
N -Onion soup and crepes suaette 
O-.'ot dogs and apple pie 

P-Frano 
Q-Dollar 
R-Pound 
S -Ruble 
T-Ten 



1, The Union of Soviet Sooisllst Republics has the largest land area 
of any oountry. 

2, France is the westernmost oountry in continental Europe, 

3, Great Britain inoludes England, Wales, Scotland, and Ireland, 
t?, The capital of the United States of America is New York, 

5, Japan has the world's most populous capital city. 



Fields ' Serial Multiple Discrimination teaching deamination Answer Sheet, Form E 



ferint last name First kiAdle 



Date Class 



fSS Beat No. 
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Table 1. It«i Analyses for SMDTE Questions over Text and Films . 





Mater- 






No. of 


Per oent passing 






Row 


ials. 


Date 


Form 


Ques. 


H 


L 


Av. 


DI 


ICC 










New Questions 










1 


Loot . 




VLS 


65 


50 


11 


30 


39 


.52 


2 


Text 




VLS 


250 


81 


28 


55 


54 


.62 


3 


Films 




VLS 


210 


79 


33 


56 


46 


.56 






Questions 


fTom Teaching Test Manuals 






4 


Text 




VLS 


370 


96 


42 


69 


54 


.70 


5 


Films 




ILS 


240 


96 


53 


75 


43 


.64 




Three Administrations of 30 "Narcotics 


" Questions 




6 


Films 11/21/67 


VLS 


30 


86 


44 


65 


43 


.51 


7 




1/9/68 


VLS 


30 


58 


18 


38 


40 


.54 


8 




3/5/68 


VLS 


30 


100 


74 


87 


27 


.51 


9 






ILS 


30 


100 


68 


84 


32 


.56 


10 






IRB 


30 


97 


45 


71 


53 


.65 






Pre- and Post-Soroening 


Exams 


over 


8 Filins 






11 


Films 


4/30/68 


VLS 


40 


50 


16 


33 


34 


.48 


12 




6/5/68 


ILS 


40 


98 


72 


85 


26 


.50 



Date under Per oent passing are for the two criterion groups i H Is the 
average acouraoy for the highest 2 7 per oent ox' the class r L is for 
the lowest 27 per cent. Av. * H + L/2. 



DI ** Discrimination Index, H - L. 

ICC ** Internal Consistency Coefficient! read from Table 3» Thorndike, 
Personnel Selection . 1949, page 348, 

VLS = verified latin square j US «= incomplete latin square i IRB « incom- 
plete randomised block. 



Exhibit B. Films Falling in Top 10 on All Three Criteria (Relevanoe, 
Teaching Effectiveness, and Interest) in Student Ratings. 



June 1968 (38 films, 356 raters) 

1. The Headbangers 

2. Gateways to The Mind 

3. Shades of Gray 

4. Aggressivo Child 



Deeenfcer 1969 (38 films, 200 raters) 

1. The Titicut Follies 

2. The Headbangers 

3. Behavior Modification 

4. Detaohed Americans 
5 Aggressive Child 

6. Shades of Gray 

7. Gateways to The Mind 

8. Bright Boy, Bad Scholar 
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Table 2. Response to New and Old Questions over Textbook and Films unde) 1 

Three Teaching Test Formats* 

No* Per oent Pass No* Per oent Pass 





Ques 


. H 


L 


Av. 


DI 


ICC 


Ques 


. H 


L 


Av. 


DI 


ICC 


4 






Textbook 




VLS Format 




Films 






New Q. 


100 


94,2 


56.4 


75.3 


37.8 


.54 


110 


91.9 


50.3 


71.1 


41.6 


.57 


Old Q. 


200 


99.5 


61,6 


80,5 


37.8 


.60 


no 


98.7 


41.4 


70.1 


57.3 


.73 














ILS 


Format 












New Q. 


175 


81.6 


27.6 


54.6 


54.0 


.62 


70 


84.7 


47.0 


65.8 


37.7 


.53 


Old Q. 


160 


98.9 


46.9 


72.9 


52.0 


.71 


150 


96.5 


45.7 


71.1 


50.9 


.68 














IRB Format 












New Q, 


no 


66.8 


18.8 


42.8 


48.1 


•59 


22 


75.7 


23.8 


49.7 


41.9 


.61 


Old Q. 


155 


91.6 


42.7 


67.1 


48.9 


.65 


130 


93.7 


53.8 


73.8 


39.9 


.58 



♦Each question answered by minimum of 157 students. 



Table 3* Item Analyses for Identloal Film Questions under Three Exam 
Formats with Different Amounts of Feedback* 

Behavior Modification (Complex Concepts) 



No. Per cent Pass 



Date 


Format 


Ques 


H 


L 


Av. 


DI 


ICC 


Conditions 


10/31/69 


VLS 


20 


35.7 


8.1 


22.0 


27.6 


.49 


New questions before seeing 
films 1 did not count. 


11/7/69 


US 


10 


95.0 


36.4 


65.7 


53.6 


.70 


After VLS and seeing filraj 
half on each W or Y form. 


12/17/69 


IRB 


20 


82.6 


36.4 


57.9 


46.5 


.54 


Five weeks after VLS and 
seeing film. 



Effect of Extra JLS Exam on IRB Scores 

Vls US IRB 

10/31 11/7 12/17 10 86.4 40*8 63*6 45*6 * 53 Half on each W or Y form, 

VLS IRB 

10/31 12/17 10 79.4 32.0 55.7 47.4 .56 Half on each W or Y form. 
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Tab]* 4, Item Analyse* for Identloal Film Question* under Three Exan 
Formats with Different Amount* of Feedback. 



Review* of Titlout Follies (Simple Concopts) 





>■ V . j 


No. 


Per oent Pass 








Date 


Format 


Ques 


t; 


L 


Ay* 


DI 


ICC 


Conditions 


11/10/69 


VLS 


10 


95.4 


45.6 


70.5 


49.8 


.66 


New questions before seeing 
films i did not oountf no time 
limit. 




12/17/69 


ILS 


10 


91.4 


69.4 


80,4 


22.0 


.41 


Five weeks after VLSj time 
limit. Half on W and Y form. 


12/17/69 


IRB 


10 


89.0 


5 '.0 


72.5 


33.0 


.4? 






Titiout Follies Film (Simple Concopts) 


11/21/69 


VLS 


20 


96.7 


75.6 


86.2 


21.1 


.43 


New questions after seeing 
filmi ocunted on grade. 


12/17/69 


ILS 


20 


97.3 


74.4 


85.9 


32.9 


.48 


Half on W and Y form. Counted 


12/17/69 


IRB 


20 


95.5 


65.7 


80.6 


29.8 


.50 


Half on W and Y form. Counted 


VLS 


ILS 
















11/21 


12/17 


10 


99.0 


77.2 


88.1 


21.8 


.50 


Half on W and Y form. Counted 


VLS 


IRB 
















11/21 


12/1? 


10 


97.6 


75.6 


86.6 


22.0 


.46 


Half on W and Y form. Counted 




ILS 

12/1? 


10 


95.6 


71.8 


83.7 


23.3 


.45 


New questions i half on V and 
Y forms. 




IRB 

12/1? 


10 


93.8 


55.8 


74.8 


38.0 


.54 


New Questions! half on W and 
Y forms. 



Exhibit C . 




Fall Quarter 
Grades 



234+ A 149 

186 B 12? 

138 C 51 

89 D 14 

88- E 7 
B/W 5 
Ino 6 



P/W6 
Total 355^ 
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Exhibit D. 



Learning about ^Programing Is a Process" from "Programed Notes" 
vs, 'Teaching Tests”, 



Left 



Odd Seats 



Experimental. 





Even Seats 



Engineering 



Entire Class 



Programed 

Notes 



Distribute on 2/11/70 

t | 

Teaching Programed 

Tests Notes 



Screen 
Programing Instr 



Teaching 

Tests 



I 



Two Filins on 2/12/70 | 

uotion, Programing is a Process 



Tost over second film on 2/13/70 I 

1 1 I 

Teaohing T-F Test T-F Test Teaohing 

Test (N ■ 84) <N * 82) Test 

(N = 83) <N = 82) 



2/12/70 (2 hours) 
Distribute Program Notes 
Soreen one film 
Disouss film 
Take T-F Test on film 
(N « 40) 



Table 5. Average Item Analyses of Total Test Scores for 120 T-F and 30 TT Items. 



True-False Test (No. right) 


Per cent passing 








H 


L 


Av. 


DI 


ICC 


Engineers, Programed Notes 


85.50 


73.08 


79.29 


12,42 


.23 


Psychology, Programed Notes 


87. C8 


71.17 


79.13 


15.91 


.2? 


Psychology, Toaching Tests 


90o0 


79.46 


84.98 


11.04 


.22 


Teaching Test (No. right) 


Psychology, Programed Notes 


94.50 


42.33 


68.42 


52.17 


.67 


Psychology, Teaohing Test 


100.00 


71.00 


85.00 


29.00 


.57 



Table 6, Significance of the Difference between Keans, 

Total Scores Psychology Engineers 

Teaching Test (Programed Notes vs. Teaching Tests) TT 6,66*** 

True-False Test (Programed Notes vs. Teaching Tests) TT 6.86*** 

(Programed Notes, Psychology vs. Engineers) ,65 

Part Scores on T-F Test 



TT - Programed Notes Vs. Teaohing Test 


TT 3 . 55 *** 




M - 


TT 9 . 49 *** 




MT - 


TT 2.60* 




MF - 


TT 4,80*** 




Film - 


.73 




New - 


TT 2.20* 




TT - Programed Notes, Psychology vs. Engineers 




.79 


M — 




1.14 


MT - 




.02 


MF — 




1.24 


Film - 




E 3 . 35 * 


New - 




.93 
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Serial Multiple Dieorinination Teaohing Exaodnation over Teaching Tests* 

(IRB Format* Use key for items 11 - 15 at bottom of page l) 

A-Eaoh question has a true-false sentenoe for its "stem"! the subject of the 
sentence should be noted* 

B-Many students had developed "entertainment sets" which interfered with the 
learning of the basic factual oontent of the film* 

C-With TF questions* the probability of a oorreot answer by ohanoe is *5 or i* 

D-An item analysis requires that the per oent of students passing eaoh ques- \ 
tion be computed for the top and bottom criterion groups, 

E-The matching associations for one question have been ordered so as to fall 
at one intersection of eaoh row and column (latin square), 

F-Students reported that films oovered so many things and "went so fast" that 
it was impossible to take satisfactory notes* 

G-One phrase or sentenoe in eaoh of four lists of matching associations is 
closely related to the key idea in eaoh true-false statement, 

H-The difficulty of the question, its discrimination index, and the internal 
consistency coefficient are then determined* 

I-If the first four matching associations for eaoh question are correct, the + 
or - symbol in the fifth column opposite the unused row is the oorreot TF 
answer* 

J-One formula frequently used to determine how much is known for sure is, "score 
** number right minus the number wrong," 

K-With four or five-alternative Multiple Choioe questions, the ohanoe faotor 
is reduced to .25 or .20 respectively* 

L-Each of the sentenoes in the lists of matching associations is always true* 
M-Even though achievement goes up signifioantly with the use of Teaching Tests, 
they still discriminate between good and poor students better than other tests* 
N-If any four questions in the same block of five have been answered correctly, 
the unused letters and symbol must be the answers to the fifth question. 

0-A oritioal and rational approaoh was sometimes difficult to develop beoause of 
over-dramatization and identification with particular actors* 

P-Even though a true-false statement may be false, it must always be associated 
with four true associations in order to obtain a oorreot answer* 

Q-If any letter or symbol has been used twice in the same block, another was 

omitted. A study of the duplicated and omitted parts should pin-point the error. 
R-If larger proportions of the classes trained with Teaohing Tests than by tra- 
ditional methods consistently aohieve the oriterion, the TT must be an effec- 
tive instructional device. 

S-The probability of narking all five parts of the easiest TT question oorreotly 
without any knowledge is only 1 out of 3,125* 

T-By studying and taking the Teaohing Tests over films at home, students knew 
what to look for when the films were soreened in olass. 

1, Description. A single Teaching Test question is oomposed of five separate 
true-false statements. 

2, Purpose . One of the objectives for developing Teaohing Tests over filrj was 
to make them teaoh effectively without using a disproportionate share of 
the total olass time. 

3* Guessing. The "guessing faotor" has been minimized or well controlled in 
the preparation of traditional objective examinations* 

4. Verification. Good students use "feedback" from the TT special answer sheet 
as they work through a VLS homework training assignment. 

5, Statistical techniques * Onoe they have been released to students, TT ques. 
tions over films become so easy that they are useless for assigning grades. 
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